The present invention relates to image processing apparatuses such as image encoding devices or image decoding devices which process an image in a predictive coding system which performs coding operation based on a correlation between frames, and more particularly, to an image processing apparatus which, even when processing a motion picture, e.g., having a high resolution, can reduce the capacity of a memory required for storing a predicting frame.
For example, there is already known an image transmission system in which an image encoding device compresses and encodes an image and transmits the compressed/encoded image, and an image decoding device receives the compressed/encoded image and decodes it.
In compressing data about a motion picture of successive frames, its compression efficiency is increased by compressing the data based on a correlation between frames preceding and subsequent to one of the successive frames. As such a predictive coding system as to utilize the preceding/subsequent frame correlation, MPEG (Moving Picture Experts Group)-2 MPEG-4 is well known.
FIG. 14 shows an example of a related art image transmission system.
The image transmission system of this example includes an image encoding device 381 for compressing data by utilizing a correlation between frames preceding and subsequent to one of frames of a motion picture, and also includes an image decoding device 382 for decoding an encoded stream into a reproducible image data. The image encoding device 381 and the image decoding device 382 are interconnected by a network 383.
In the image encoding device 381, an A/D converter 392 receives a video signal issued from a video input device 391 such as a camera and converts the received signal to digital image data. The digital image data issued from the A/D converter 392 is stored in a frame memory 393. For example, the frame memory 393 divides a frame into square blocks of 16×16 called macro blocks in MPEG-4 or AVC (Advanced Video Coding), and then outputs the divided macro blocks to an encoder 394 on a macroblock-by-macroblock basis.
When a predictive frame is stored in a predictive frame memory 396, the encoder 394 performs encoding operation based on a correlation between preceding and subsequent frames. When a predictive frame is not stored in the memory, the encoder 394 performs encoding operation based on a correlation between adjacent pixels in a frame. Even when the predictive frame is stored in the memory, the encoder 394 can perform the intra-frame correlation encoding operation.
The encoder 394, after receiving data to be encoded, performs predetermined encoding operation in the aforementioned manner and outputs the encoded stream to a stream buffer 397. A decoder 395, when receiving the encoded stream, decodes the encoded data into decoded image data and stores the decoded data in the predictive frame memory 396.
The stream buffer 397, when receiving the encoded stream, outputs the encoded stream to a communication controller 398. The communication controller 398 transmits the encoded stream to the image decoding device 382 via the network 383 by a predetermined communication procedure such as TCP/IP.
In the image decoding device 382, a communication controller 401 receives the encoded stream from the image encoding device 381 and outputs the received encoded stream to a stream buffer 402. The stream buffer 402 outputs the encoded stream to a decoder 403. The decoder 403 decodes the received encoded stream into decoded image data, and stores the data in a memory 405. The decoder ¥403 also stores the decoded data in a predictive frame memory 404. When the frame to be decoded is already encoded by the predictive coding, the decoder 403 decodes the frame with use of predictive frame data stored in the predictive frame memory 404.
Image data issued from the memory 405 for receiving the decoded image data is converted by a D/A converter 406 to an analog signal. The analog signal is reproduced by a video output device 407 such as a monitor.
As one of such related arts, JP-A-8-242446 is proposed.
Such an image encoding device 381 and an image decoding device 382 based on the predictive coding system as shown in FIG. 14 require the predictive frame memories 396 and 404, and the necessary sizes of such memories depends on the number of pixels in a motion picture and on the number of frames to be stored for prediction.
In these years, however, as the performance of an imaging element is made high, an increasing number of motion pictures have very high resolutions exceeding 1,000,000 pixels as in a HDTV (High Definition TeleVision). The resolutions of even motion pictures to be encoded by the image encoding device 381 are correspondingly made increasingly higher. This involves a problem with the increased necessary capacity of the predictive frame memories 396 and 404.